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Abstract 

The Bethe approximation, discovered in statistical physics, gives 
an efficient algorithm called belief propagation (BP) for approximat- 
ing a partition function. BP empirically gives an accurate approx- 
imation for many problems, e.g., low-density parity-check codes, 
compressed sensing, etc. Recently, Vontobel gives a novel char- 
acterization of the Bethe approximation using graph cover. In this 
paper, a new approximation based on the Bethe approximation is 
proposed. The new approximation is derived from Vontobel's char- 
acterization using graph cover, and expressed by using the edge zeta 
function, which is related with the Hessian of the Bethe free energy 
as shown by Watanabe and Fukumizu. On some conditions, it is 
proved that the new approximation is asymptotically better than the 
Bethe approximation. 
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I. Introduction 

Calculation of partition functions is one of the central 
problems in statistical physics. The Bethe approximation is 
an empirically successful efficient approximation for many 
problems Q. A few results about asymptotic exactness of the 
Bethe approximation are recently known. Recently, Vontobel 
shows a novel characterization of the Bethe approximation 
using graph cover |2|. In this paper, a series of generaliza- 
tions of the Bethe approximation is shown from Vontobel's 
characterization of the Bethe free energy via asymptotic ex- 
pansion. 

II. Factor graph and the Bethe free energy 

Let A" be a finite set. A factor graph is a bipartite graph 
consisting of variable nodes and factor nodes. A factor graph 
including variable nodes defines a probability measure on 
X'^. The set of variable nodes, the set of factor nodes and the 
set of edges are denoted hyV,F and E CV xF, respectively. 
The neighborhoods of a factor node aGF and a variable node 
/ e y are denoted by C y and di C F, respectively. The 
degrees of variable node / and factor node a are denoted by 
di and da, respectively. For each factor node a, there exists 
a corresponding non-negative function /„ : X'^" K>o- The 
probability mass function defined by the factor graph is 



' aeF 



The constant Z for the normalization is called the partition 
function. The calculation of Z is generally a #P-complete 
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problem. Approximating Z is one of the central problems of 
statistical physics and information theory 1 1]. 

In the following, the Bethe free energy is defined using the 
variational method |3|. Let p{x) be a probability mass func- 
tion defined by a factor graph. From the following equation 
for any probability measure ^(jc) on X^, 

D{q\\p):= ^ q{x)\0g^ 

==iogz-^ E ?(-«^)iog/«(-«^3fl)+ E ?(-«^)iog?(-«^) 

logZ + Z^Gibbs(?) -'HGibbs(?) -■ logZ-hFGibbs(?) 

one obtains the equation — logZ = /^Gibbs(p) = 
min^gP^;^.^-) fGibbs('?) where V{X^) denotes the set of 
probability measures on X'^ . The Bethe free energy, defined 
in the following, is an approximation of the Gibbs free 
energy, which is FcibbsC?) in the above equation. 

Definition 1 (Bethe free energy). For {bi e and 

{ba e V {X''")} adF satisfying the reducibility 

E ba{x3a)^bi{z), yii,a)eE (1) 

X3„eX''a,Xi=z 

the Bethe free energy is defined as 

FB^th.{{bi},{ba}) ■■=UB,±e{{b,},{ba})-nBeth^{{bi},{ba}) 

where 

Z^Bethe({^},{^«}) := - E E baiX3a)iogfaixSa) 

"•^F xg^ex''" 

'HBethei{bi},{ba}) := - E E l^ai^da)^°S.b"i^da) 

+ E('^'-i) E ^'(^/)iog^«(x,). □ 

iev xiex 

Since the Bethe free energy is considered as an approxi- 
mation of the Gibbs free energy, the minimum of the Bethe 
free energy is regarded as an approximation of — logZ. The 
stationary condition of the Bethe free energy is 



bi{x) = 



1 



Zi{{ma^i}aedi) „ei 
1 



Yl ma^i{x) (2) 
ma->i{x)mi^a{x), yaedi. 



where {('Ma^/,'«,-^a) S ^('^)^}(i,a)G£ satisfies the belief 
propagation (BP) equations 

mi^aix)oc Y[ mh^i{x) 

(3) 

ma^i{x)°^ E f"(^) n ^i^cAxj) 

xeX''o,Xj=x j£da\i 



2 



and where Za{{mi^a}ieda)^ Zi{{ma^i}aedi) and 
Zi.a{'na^i,nii^a) are the constants for the normahzations 

Zi{{ma^i}^,^3i) ^ n m„^i{x) 

xeXaedi 

xex 

By substituting (|2]l to the definition of the Bethe free energy 
and using Q, one obtains the representation 

^Bethe({'«i^a}, {'Wfl^i}) = - ^ logZ,({m„^,}„g5,) (4) 

iev 

aeF (i,a)eE 

Note that the stationary condition of (|4]l is also given by the 
BP equations Since the Bethe free energy is an approx- 
imation of — logZ, the Bethe partition function is defined as 

ZBethe({«^(^a}, {'Wa^;}) := exp{-fbethe({'Wi^a}, {'Wa^/})} 

i.e., 

ZBethe({'«/^«},{'«a^/}) = Y\Zi{{ma^i} aedi) 



using a set of functions {f^. : X 
cient statistic. Here, 



rk=\.2 d 



called a suffi- 



1 



■Y{Za{{m,^a}ie3a) 11 „, ^ 

for {{ma^i,mi^a)} {i,a)eE satisfying (O. We also use the 
notation Zeethe := min{„,.^j {,„^^.} ZBethe({»J;^a}, 
We assume in this paper that all variable nodes have the same 
alphabet X only for the simplicity. All results in this paper 
can be generalized to the case that each variable node / e V 
has distinct alphabet A", . 

III. Exponential family 

In this section, exponential family of probability distribu- 
tions is introduced. The exponential family is a class of para- 
metric family of distributions. A family of probability dis- 
tributions having a parameter e © C M'' for some c/ e N is 
called a parametric family of distributions. In this paper, the 
existence of the first and the second derivatives of probability 
density (mass) function p{x\6) with respect to the parameter 
is assumed. 

Definition 2 (Fisher information matrix). For a parametric 
family of distributions, the Fisher information matrix is de- 
fined for each parameter e © as 



diogp{x\e)diogp{x\0) 



do, 



where X ^ p{X \6) and where 5^ / denotes the {k, Z)-element 
of a matrix B. 

Definition 3 (Exponential family). The exponential family is 
a parametric family of probability distributions whose proba- 
bility density (mass) functions can be expressed in the form 

p{x\0)=expl^Ytk{x)Ok^xi/{0)'^ 



\l/{0) log ^ exp <^ Y tk{x)9k 

The parameter 9 of an exponential family is called 
a natural parameter The parameterization with {rjk := 
E[tii{X)]}ii=i d is also useful, and the parameter is called 
an expectation parameter 

Example 4 (Multinomial distribution). The family of dis- 
tributions on a finite set X = {0,1,...,^— 1} can be re- 
garded as the exponential family with a sufficient statistic 
{tx{x') — Sx^x'}xex\o- In this case, rj^ = p{x \ 0) foix E X\0. 

Let V2g(6/) be the Hessian matrix of Q at 0. Let Vai-[f(X)] 
be the covariance matrix of {fjt(^)}A:=i d with respect to 
p{x I 0). Then, it is easy to see that Var[i(X)] = J{0). 

Lemma 5. It holds J{0) — J{r}Y'^ where J{0) and J{t]) 
are the Fisher information matrices with respect to the nat- 
ural parameter and the expectation parameter r}. Further- 
more, it holds -V^H{i]) = J{r}) and V^\if{0) = J{0) where 
H{i]) := — Y.XGX p{x I ^)^ogp{x I T}) is the Shannon entropy. 

IV. Characterization of the Bethe free 

ENERGY USING THE LOOP CALCULUS 
A. Linear transforms of the factor functions 

In this section, a characterization of the Bethe free energy 
is described which is obtained by Chernyak and Chertkov |4 1 . 
Assume that a function fa{x) : X''" — > R>o is represented by 
using /„ : y''^ R>o and : A" x 3^ ^ M>o as 

fa{xda)= E /"(3'5J n 

yaaeyd" ieda 

for some set y. Let 5 (x, z) be a function which takes 1 if x = z 
and takes otherwise. When y = X, this representation is 
obtained by letting 

fa{yda)= E n ^'>(^''-^') 

xg^eX^a ieda 

for {{^i,a,^i,a)}ieda Satisfying 
yex 



or equivalently 



E ^i,a{y,x)(pi,a{x,w) = 5{y,w) 

xex 



(6) 



The partition function can be rewritten by using the trans- 
form (|5]) as 

z= E Y{Uyda)Y{\YY{^Ux.yi.a)\. (d 

y^y\E\aeF ieV\xeXaedi / 

In this representation, the variables y e y^ are associated 
with edges of the original factor graph. Both the variable 
nodes and factor nodes in the original graph can be regarded 
as factor nodes in the new representation Q. The transform 
of representation gives a unified way to understand many 
equations, e.g., the Mac Williams identity |5j|. 



B. Bethe transforms and characterization of Bethe free en- 
ergy 

Let y — X. We now consider the following additional con- 
ditions on )}(;,„) G£. For each / e V and a£F, 



E f«i^da) n 4«(3'i:-^i) = 0, 3\j e da,yj ^ 0. 



(8) 



On these conditions, if the subset {{i,a) E E \ yi,a 7^ 0} C £ of 
edges generates degree-one variable nodes or degree-one fac- 
tor nodes, the term in O corresponding to y is zero. Hence, 
in dTjl, we only have to take the sum overj g A" 1^1 satisfying 
S E I ^ 0} S G where G is the set of generalized 
loops defined as G := {£' C £ | do{E') ^ l,Vo G V UF}. 
Here, di{E') := \{{i,a) eE'\ae di}\ unddaiE') := \{{i,a) £ 
E' I / G (9fl}|. The conditions (|6|l and (l8]i are equivalent to the 
condition (|6]l together with 



kai^^x)^- n 0,>(X,O) 



(9) 



For {(wa^i,?M,^a)}(,,j)££ which satisfies the BP equa- 
tions (0,:„(x,O),0;,„(O,x)) = {ci.ama^i{x),Ci,ami^a{x)) 

provides a solution of (|9]i where ciaCia = 1 /Zi,a{nii^a,nia^i)- 
In this case, the contribution of all-zero assignment in (|7]i is 
the Bethe partition function ZBethe({'Wi'^a}, {'Wa^^})- 

For the binary case i.e., A" = {0, 1}, 1), = 

((-l)''c;.„m;^„(i),(-l)^Ci,a»Jfl^.i(-^)) Satisfies (|6]l. In this 
case, one obtains the following lemma by substituting the 
above values of {^i.a, ^,,u}(,',„)g£ to (|7]i. Let be an ex- 
pectation with respect to a probability mass function p. 

Lemma 6 (0, J?)). Assume that the alphabet is binary, i.e., 
X ~ {0, 1}. Letmi := {Xi)i,. =bj(l). For any stationary point 
({bj},{ba}) of the Bethe free energy. 



ZBethe({M,{^«}) ^ 



(10) 



where 



Z{E'):=Y\ 



Xi — nii 



di(E') 



iiGF \ieda,{i.a)eE' V ((^i ^ / 

For non-binary cases, the conditions ^ and (O do not fix 
^i.a)}(/,a)G£ Uniquely. In |4|, it is suggested to use loop 
calculus iteratively for each Z{E'). In this paper, on the other 
hand, we propose an explicit choice of {^i,cn^i.a}- As shown 
in Example]?] the family of multinomial distributions can be 
regarded as an exponential family. Let 0, and i/,- be a natural 
parameter and an expectation parameter of bi, respectively. 



Then, 0,- ,j(x,y) and ^j-.„(jc,y) forx G X andy G A'\0 are cho- 
sen as 

<^iAx,y) _d\ogbi{x) ha{x,y) _d\ogbi{x) 



The partial derivatives in the first and second equations in 
the above are those with respect to the coordinate systems 
{j?(,v}yGA'\o and {0,:v}yga'\o. respectively. One can easily 
confirm that these {0,-.a, ^;.a} satisfy the condition Q as fol- 
lows. For w G A" \ 0, it holds 



E ka{Q,x)<Pi.aM = E ^W^^^^ =0. 

xex xex 



Similarly, Zxex ha (y,-^)^;, a (x, 0) = for any y G A \ 0. For 
y,w e X\0, it holds 

dlogbi{x) dlogbi{x) 



E kaiy,x)(j)i,a{x,w) = E ^'■(■«) D„ 

xex xex "^',y 



dGi 



dbi(x) r , , , 

= L^^hAx)-r^t,.] 

xex "^'y 

dr]i^„ ^ dbi{x) 

Then, one obtains the following theorem. 

Theorem 7 (Loop calculus for non-binary alphabet). For any 

stationary point ({/?,}, {^a}) of the Bethe free energy, dlOl l 
holds where 

^y^')- E n( n '-^^) 

ye(x\{Q})\'^'\''^V \aedi,(i,a)eE' "'IhyiM / 



■n n ^ 



This theorem is useful for understanding the approxima- 
tions to be proposed in Sec.lVIl 

V. Edge zeta function and the determinant 
OF Hessian of the Bethe free energy 

A. Edge zeta function 

In this section, the edge zeta function and Watanabe- 
Fukumizu formula are introduced [SJ . 



Definition 8. For (/,«) G E and {j,b) G E, (; 



u 



b) 



def 



i G da,i 7^ y',fl 7^ b. 



Definition 9 (Prime cycle). The closed walk e 1 ^ 62 ^ • • • ^ 
Cn ^ e\ is said to be a prime cycle if and only if it cannot 
be expressed as a power of another walk. Prime cycles are 
identified up to cyclic permutations. 



Definition 10 (Edge zeta function fSj). Let ri^a be a natural 
number associated with an edge (/,«) G E and M(,-^a),(;^i) be 
an ri^a x fj^t matrix for (/ a) ^ [j — >■ b). Then, the edge 
zeta function is defined as 

1 



C(«) 



n 



p={ei^e2--^en^e\) det ily^ 
is a prime cycle ^ 



I prime cycle 

where Ir is the identity matrix of size r. 
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If a factor graph includes more than one cycle, the number 
of prime cycles is infinite. Hence, it is difficult to evaluate 
C,{u) from the definition. The following lemma is generally 
useful for evaluating C(m). 

Lemma 11 (Bass's formula). It holds C,{u) = det(T|£| — 
M.{u))^^ where is the identity matrix of size \E\ as 
a block matrix, and where Ai(u)ggi :— u^^i if e e' and 
Ai(u)gy := otherwise. 

Furthermore, the other expression of ^(h) is known on 
some condition. 

Lemma 12 (Ihara-Bass formula fSl). Let r,- be a natural num- 
ber associated with a variable node i G V. When U(i^a).(j^b) 
is an ri x rj matrix independent ofb and denoted by uf^j, 

aeF 

where V is an N x N block diagonal matrix defined by Vij := 
dilri, where hi" is a daX da block matrix defined by lAfj 7^, 
and U" J := for i ^ j, and where W is an N x N block 
matrix defined by Wij :— 'E.a:{i.j}cda^'i^j- Here, w"^j :— 

B. Determinant of Hessian of the Bethe free energy 

In this subsection, {bi] and {ba] are assumed to be mem- 
bers of an arbitrary fixed parametric family of distributions. 
The alphabet X is not necessarily finite. For / G V, bj 
has a parameter ?/,. For a G F, ba has a parameter i/„ = 
('?(a>j {li)i€da)- The Condition ^ is assumed to be satisfied 
for any coordinate {{Ui}An{a)})- I" the following, a param- 
eter Tj is denoted by the normal font T] for the simplicity. For 
/ £ V and a G F, let Hj and //„ be the Shannon entropy of bi 
and ba, respectively. The notation B y means that a matrix 
B is positive definite. 

Lemma 13. For ({i?(},{i?(a)}) satisfying 



dHi 



> 0, V/ e V 



dH„ 



driidrii 
it holds that 

det(V2FBethe({j?,}, {!?(„)})) 

dH, 



dri{a)dri(^a) 



^0, Vfl e F 



ndet( 



iev 
where 



ridet 

aeF 



dr\(a)dr\(a) 



( dHi 



\driidrii 
dH, ( dH, 



E 



-1 



<9t7,<5t7(„) \dri^a)dri^a) 
Proof It is easy to see that 

<5^Bethe dH, 



dHa 
driidrij 

dHg 
dri{a)dilj 



dHj 
drijdrij 



driidrij 

dri{a)dri(b) 



y 



Sij{di-l) 



dHi 
driidrji 



^5, 



dHa 



dF, 



Bethe 



dHa 



dr\(a)dr\(b)' di]idi](a) driidrj^a)' 



Let V be a block diagonal matrix defined by 

dH , . dH, 



Va 



driidr]i' drii^a)dri{a) 
andC := V^FBethe({j?i}7 {j?(a>}) ^ V- Then, one obtains 

WBethe({j7,},{%)}) = vi(I„+|f|+V-2CV-i)V2 

For J" := V" 2 CV" i , it holds that 
dHa 



Si.jdilrj, 



dHa 



-VaJ 



a,i — ya,a 



:Fa.b - 

dHa 



-V: 



driidri(^a) ^J?(«>^J?; 

From det(V2FBe,he({i7,},{J7(„>})) = det(V)det(I^+|f , +^) 
and 

det(V) = Hdet (^) n det (t"^ 
fev ^ driidrii J ^f^j, ^dri(^a)dri{a) 

we only have to prove det(I^_|_|/r| + J-) = det(lAr — V + Q). 
For u X u, u X V and v x u matrices A, B and C, respec- 

C I, 



tively, it holds 
det 









'A BC B 




-c 




ly 



and hence 



A B 
C /, 



= det(A - BC). Therefore, 



det (2^+ 1 f I +T) = det(ZAf + JVv - -T^vf-^vf) 
= det(ZA,-2? + g). 



□ 



For an exponential family, the determinant of Hessian of 
the Bethe free energy is connected to the edge zeta function. 

Lemma 14 (Watanabe-Fukumizu formula E]). Let {rj,} and 
{rja} be the expectation parameters for {bi} and {ba}, re- 
spectively. Let {?,} and be the sufficient statistics for 
{bi{ and {ba}, respectively. Then, it holds 

C(«)-^=det(VVBethe({l7,},{l7(„)})) 

• n det( Varfo, [f (X, )] ) ' ' Y\ det( Var,„ [f, (X^ J] ) - ' 

iev aeF 

where ri is the number of parameters ofbifor i € V and where 

uUj = covhMXi)MXj)] 

:= Vari, [f, (X, )] " 2 Cov,^ [f, (X,) , tj (X,)] Var,^ [tj (Xj)]-^. 
Here, Cov h^[ti{Xi),tj{Xj)] is a matrix whose (k,l)-element is 
E[{ti,kiXi)-E[ti,kiXi)])itjjiXj)-E[tj,iiXj))]. 

VI. The new approximations via asymptotic 

EXPANSION 
A. The series of approximations 

Recently, a new characterization of the Bethe free energy 
is shown by Vontobel 121. Let Z(M) be the random variable 
corresponding to the partition function of a uniformly chosen 
random graph cover of the original factor graph where M is 
the number of copies of the original graph in the graph cover. 
See [2 1 for a detailed definition of a graph cover. Then, the 
Bethe free energy appears naturally as follows. 
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Lemma 15 (H). It holds E[Z(M)] = z'^^t'^^ asM^oc. 

This result gives a new characterization of the Bethe free 
energy. More detailed analysis is obtained as follows. 

Lemma 16 (f9\). Let S{f) be the support of a function f. Let 
B be the set of minima o/^Bethe({^(}, {^a})- Assume \B\ < °° 
and 

b*{x) > 0, Vx e X, yi e y, b*{x) > 0, Va: e yaeF 
det(vVBethe({J7r},{4,)})) >0 
for all i{b*},{b*}) G B. Then, it holds that as M °° 

E[Z(M)] ^Z^'.^he E ( det(v2FBethe({i7;},{T?*„)}) 

({fo*},{foS})GB V 

•nn^*w'"''n n 

iev.xex cieFxeS(fa) / 



({i*},{6S})G6 

where m"^^ = Cor;,* ant/ where TJ; = 

{^WKga-VO an'^ '?(£,) = Ta{{ba{x)}j,^s,^f^)). Here, for 
a € F, Ta{ri) denotes an any subset ofrj with which {T7,},ec)a 
can be regarded as an expectation parameter for ba- 

Furthermore, except for cases including singularity, the 
expectation of a partition function of a graph cover has the 
asymptotic expansion 



logE[Z(M)]^MlogZBethe + E 

k=0 



8k 



where go := \ogY.{{hl},{bi})&B vCT") and where {gk}k=\x.-. 
are some constants similarly to the Edgeworth expan- 
sion [10|. Based on the above divergent series, we demon- 
strate the following series of approximations by letting M — 
1, which may seem shameful 

logZ « logZBethe 
logZwlogZBethe+^O 
logZ « logZBethe +gQ + gl 



Definition 17 (Asymptotic Bethe approximation). For m = 
0,1,..., the asymptotic Bethe approximation of order m is 
defined as 

^AB :=ZBetheexp<^ ^ gk 

B. Asymptotic exactness of the asymptotic Bethe approxi- 
mation of order 1 
In this subsection, we show cases in which the asymptotic 
Bethe approximation Z^g is asymptotically better in some 
limit than the Bethe approximation ZBethe- Let the set L2 of 
loops be 

L2 := {e' QE\ E' is connected 4o{E') or 2, 

VoeVUF}. 



The set L2 of loops is a subset of the set G of generalized 
loops. For the binary case, a contribution Z{E') in Lemma|6] 
of a loop E' € L2 is 

Z{E') = Cor,^^, [X,-,„X„ ] n Cori„^ [X,, ] 

A:=l 

where {ii^ — ^ Of) ^ {i\ — > oi) (/2 — > 02) • • • ^ (/(' — > ai) 
forms the loop E' and where i:—\E'\. In the above, weight 
Cor;,^ [Xi^Xj] of the connection of edges (4 -> fl<.) ^ {ik+i ~^ 
fljt+i) is the same as in the edge zeta function used 

forz['i| for A:= 

For non-binary cases, the partition function of a single- 
cycle graph can also be calculated easily by using the well- 
known method of the transfer matrix. 

Lemma 18 (Transfer matrices). 

E /iv(xN,xi) n/K^.-,^/+i) =tr(F(i)FP)...FW) 

where F''^ is a \X\ x \X\ matrix with F^'^ = f{x,x'). 

From this lemma and Lemma|5] one obtains Z{E') in The- 
orem|2]for E' E L2 as 

Z (£') = tr (Cor„„^ [f„ (X,-, ) , f,, (Xt, )]Cor,^^ (X.^ ) , f,, (X,-, )] 
•••Cor,„Jf,-,(X,),f,-,(X,-,)]). (11) 

First, we consider the simplest non-trivial example, 
namely, a single-cycle factor graph. 

Example 19 (Single-cycle factor graph). For a single-cycle 
graph, the Bethe free energy is convex with respect to the 
expectation parameters and hence the stationary point is 
unique IS]. For the unique solution {b*f},{b*}, one obtains 
from Theorem|7]and (fTTT i that 



ZBethe({^*},{^^}) 



l+tr(A) 



where 



A -.^ Covb,^ [fi (Xi),f2(X2)]Cor;,.^ [t2{X2)MX3)] 

•••Cori.^[fA,(XA,),fi(Xi)]. 

On the other hand, the square root of the edge zeta function 
is 

1 



Fromdet(/|;t.|_i-A) ^ l~tT{A) + 0{p{Af) asA->0, where 
p (A) denotes the spectral radius of A, one obtains the follow- 
ing asymptotic equality 



1 



l-tr(A) + 0(p(A)2) 
l+tr(A) + 0(p(A)2) 



^Bethe 



-OipiAf 



Hence, y%(H) is an accurate approximation for Z/ZBethe 
whenever the matrix A is close to zero. □ 
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For a factor graph including multiple cycles, we consider 
the case that the functions {/„} have a parameter j3 > 0. We 
assume that there is a unique minimum of the Bethe free en- 
ergy. Furthermore, we assume that Z/Zeethe ^ 1 as /3 0, 
and that the contribution of loops dominates Z/Zeethe, i-e., 

-^ = 1+ Z{E')+o( (12) 

^Bethe E'eL2\{0} ^E'eL2\{0} ' 

as j3 — > 0. On the other hand, one obtains 

£'GL2\{0} \£'GL2\{0} ' 

Hence, the square root of the edge zeta function ^ ^{u) gives 
the dominant terms in Z/ZBethe in the limit /3 — > for an arbi- 
trary topological graph. The derivation of the above equation 
is omitted due to the lack of the space. 

Example 20 (High-temperature expansion for the Ising 
model). For Ji j G M. and /3 > 0, the partition function of the 
Ising model is 

Z= £ exp|j3( E Ji.jXiXj + Yh.xM. (13) 

xe{+L-l}N ^{iJ)eF iev /J 

When hj = for all / g V, the set of uniform messages is the 
trivial solution of the BP equations (O. Although it is not the 
unique solution of (O, if j3 is sufficiently small, the set of uni- 
form messages is the unique minimum of the Bethe free en- 
ergy. The Fourier transform is equivalent to the Bethe trans- 
form for the uniform messages. From Lemma |6] or equiva- 
lently from the MacWilUams identity, one obtains 

Z = 2^ n cosh(j37,j) E n tanh(i3y,,). 

(i.j)eF E'CE, (i.j)eE' 

di{E') is even 

In this case, the Bethe free energy evaluated with the uniform 
messages is 2^n(i,;)Gf cosh(j37j-,/). Since ( fT2] l is satisfied in 

this case, Z^g is a more accurate approximation than Zflethe 
in the limit j3 — > 0. When hj ^ 0, the uniform messages are 
not a solution of the BP equations Q. In that case, from (|2]l, 
Cor^^ [^(■j'^;] is equal to 

sinh(2/3y,j) 

Y/cosh(2/,) +cosh(2/3y;,/)-\/cosh(2/j) +cosh(2/3y,-.;) 

where mi^a °= exp{Z,} and mj^a °= expj/j} jSj. Since 
jCor^^ [^(■j-'^;] I takes the maximum | tanh(j37,j)| at /,■ = Ij — 0, 
as j3 — > 0, Covi,^\Xi,Xj] — > and hence, (fT2l) is satisfied. On 
the other hand, as hi °o, it also holds Cor^ JX/.X,] 
since /,■ ±°o as /i, ^ ±°o. Hence, the condition ( fT2] i is also 
satisfied in the limit hj ±°o for all / e V. □ 

Remark 21. Let us consider the (not bipartite) graph in 
which a degree-two factor node in the Ising model ( flj] ) is 
replaced by an edge. Due to Kac and Ward ifTTl . if the 
graph is planar and if hi = for all / G V, the partition 
function Z can be calculated in polynomial time from the 



beautiful equation Z — Zsethe/ vCT") where U{i^a).{j^b) = 

tanh(/37,:y)exp{v^7(;^«).(i^/^)/2}- Here, 7(;^^)^(j-^i) de- 
notes the angle of the edge connection (/ — >■ a) ^ {j — >■ b). 
This equation exists only when the graph is a certain type of 
planar graph and the magnetic field hi is zero for all / € V. 

□ 

Even if each Cor/, JX, ,Xy] does not go to 0, if the product 
of them along a loop goes to 0, the condition (fT2l) can be sat- 
isfied. Some sparse factor graphs can satisfy this condition in 
the large-size limit N ^ °°. The justification of the approxi- 
mation zj^g for some sparse factor graphs is an open problem. 

When the Hessian of the Bethe free energy is not positive 
definite at some critical temperature /3c, the edge zeta function 
diverges. This situation is considered as a finite-size analogue 
of the second-order phase transition. Similarly, if the minima 
of the Bethe free energy discontinuously jumps at )3c, Z^g is 
discontinuous at )3c for m > I. This situation is considered 
as a finite-size analogue of the first-order phase transition. In 
these cases, it is better to consider another limit for j3 around 
j3c, e.g., 5 = {p- Pc)/N''' is fixed for some c> 1 12]. 

For the three-body Ising model without magnetic field, 
^y^Ju) — 1 for the uniform messages and hence zj^g = 

(2) 

ZBethe- We Can intuitively guess that Z^^^g takes account of 
contributions of the set of connected generalized loops in 
which all degrees must be at most three since the derivation of 
gi requires derivation of third-order derivatives of the Bethe 
free energy, which includes the third-order statistics in Theo- 
rem |7] This idea can be generalized to zj^g for any m> I. 
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